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ABSTRACT 



The purpose of this guide, developed by the Association of 
American Medical Colleges group on Graduate Research, Education, and Training 
(GREAT) , is to outline a model process by which graduate programs can measure 
program success and that can be adapted to particular circumstances and 
priorities. The first sections of the guide examine the history and mission 
of graduate education in medical schools and outline goals and objectives 
-that -v±rtriaLly -alJ. programs aim to achieve. The guide also sets out five 
steps for achieving success: (1) defining the program goals and objectives, 

(2) pursuing activities that promote attainment of goals and objectives, (3) 
identifying indicators of success, (4) data gathering, and (5) implementing 
change. A final section of the guide presents six self-assessment survey 
instruments: a qualitative questionnaire for self-evaluation of graduate 
programs; assessment of faculty and Ph.D. program characteristics; assessment 
of admissions and graduate student populations by program; faculty assessment 
of recent graduates; survey of student career goals and satisfaction; and 
aggregated data on student career goals and satisfaction. Appended are tables 
of sources of data for graduate program self-assessment and an annotated list 
of resources. (CH) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



Benchmarks of Success in Graduate Programs 



http://www.aamc.org/about/gre/selfasst.ht 



AA^ I About the Amcf: 



r. Graduate Research, Education, and 
5 Training (GREAT) Group 



Self Assessment of Graduate Programs in the Biomedical Sciences 













U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

Om his document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



O 

o 

ERIC 



BEST COPY AVAtLABLE 






Benchmarks of Success in Graduate Programs 



http://www.aamc.org/about/gre/selfasst.htm 




Graduate Research, Education, and 
Training (GREAT) Group 



Self Assessment of Graduate Programs in the Biomedical Sciences 

A key feature of the 1998 GREAT Group annual conference was a session 
devoted to the work of the GREAT Group Task Force on Benchmarks of Success 
in Graduate Programs. This Task Force was appointed by the GREAT Group 
membership following discussions taking place at earlier GREAT Group meetings 
on the value of having graduate programs periodically conduct a formal process of 
self-assessment to discern the extent to which they are meeting their goals. 

Program directors also expressed the desire for resources to assist in the task of 
defining goals and identif^ng the indicators that allow assessment of whether 
their objectives are being achieved. 

The Task Force on Benchmarks of Success in Graduate Programs worked over the 
course of a year to develop such a resource, which is now available electronically. 

The purpose of the guide is to outline for graduate programs a process by which 
they can develop and measure their own "success," however they choose to define 
it. This guide is not prescriptive and thus does not impose on programs any 
particular criteria or approach. Rather, it is suggestive, pointing to a wide array of 
possibilities and outlining a model process that users of this guide should adapt to 
their own particular circumstances and priorities. 

A companion to this narrative guide is a set of survey instruments. These are tools 
to allow programs to conduct qualitative and quantitative assessments of their own 
programmatic and educational characteristics. You are encouraged to utilize the 
surveys at your own institution. 

It is important to note that this project is not a prelude to further accreditation of 
graduate programs, most of which are already accredited by state, regional, or 
local accrediting bodies. Nonetheless, the AAMC and the Liaison Committee on 
Medical Education (LCME) do encourage self-evaluation for the purposes of 
maintaining the quality of Ph.D. education, and thus this document was conceived 
of and prepared by the GREAT Group as a resource to its own membership. Input 
on this guide and other aspects of this project are welcome and may be addressed 
to the GREAT Group Executive Secretary . 

HTML version 




PDF Version 



BEST COPY AVAILABLE 



er|c 



1 of 2 



3 



9/13/00 3:21 PM 




Benchmarks of Success in Graduate Programs 



http://www.aamc.org/about/gre/selfasst.htm 



iGetAcrobat^^ 



To view and print documents in Portable Document Format, you will need the Adobe Acrobat® 
Reader, which can be downloaded from this site. 



Acrobat® Reader Copyright © 1987-1996 Adobe Systems Incorporated, All rights reserved, Adobe 
and Acrobat are trademarks of Adobe Systems Incorporated which may be registered in certain 
jurisdictions. 




Home I About AAMC | Advocacy I News Room I Student & Applicant Info. | Information Resources 
Meetings | Medical Education 1 Research | Health Care 



Revised: 20 March 2000 
Please send us your questions and comments, 
© 1995-2000 AAMC Terms and conditions. 
Privacy Statement 




4 



2 of 2 



9/13/00 3:21 PM 






Self-Assessment of Graduate Programs 
in the Biomedical Sciences 

Narrative Guide and Companion 
Survey Instruments 



Report of the Task Force on Benchmarks of 
Success in Graduate Programs 
AAMC GREAT Group 

June 1999 







5 



Table of Contents 



I. Preamble 1 

II. The History and Mission of Graduate Education in Medical Schools 3 

ITT . The Goals and Objectives of a Successful Graduate Programs 3 

IV. How Programs Can Work Toward Achieving Success 5 

Step 1: Defining Program Goals and Objectives 5 

Step 2: Pursuing Activities That Promote Attainment of Goals and Objectives . . 6 

Step 3 : Identifying Indicators of Success 7 

Step 4: Data Gathering 7 

Step 5: Implementing Change , 8 

V. Self-Assessment Surveys 9 

Surveys: 

Qualitative Questiormaire for Self-Evaluation of Graduate Programs 11 

Assessment of Faculty and Ph.D. Program Characteristics 13 

Assessment of Admissions and Graduate Student Populations by Program 19 

Faculty Assessment of Recent Graduates 28 

Survey of Student Career Goals and Satisfaction 24 

Aggregated Data on Student Career Goals and Satisfaction 26 

Tables: 

Existing Sources of Data for Graduate Program Self-Assessment 

Table 1. Assessment of Faculty and Program Characteristics 30 

Table 2. Assessment of Graduate Student Populations 32 

Table 3. Assessment of Recent Graduates 34 

Other Resources 35 



i 



ERIC 



6 



SELF-ASSESSMENT OF GRADUATE PROGRAMS 
IN THE BIOMEDICAL SCIENCES 

NARRATIVE GUIDE AND COMPANION SURVEY INSTRUMENTS 



I. Preamble 

The AAMC Group on Graduate Research, Education, and Training (GREAT Group) is a 
professional body of approximately 260 individuals with a responsibility for, or interest in, Ph.D. 
training that occurs in the medical school setting. At its 1 996 and 1 997 conferences, the 
membership of this group recognized the value of having graduate programs periodically conduct 
a formal process of self-assessment to discern the extent to which they are meeting their goals. 
Participants at these meetings also expressed the desire for resources of which program directors 
could avail themselves when taking on the task of defining goals and identifying the indicators that 
allow assessment of whether their objectives are being achieved. In light of this, the GREAT 
Group membership called on its Steering Committee to appoint a task force to develop just such a 
resource. 

The Task Force on Benchmarks of Success in Graduate Programs was thus assembled and 
worked over the following year on this narrative guide. The purpose of the guide is to outline for 
graduate programs a process by which they can develop and measure their own benchmarks of 
“success,” however they choose to define it. This guide is not prescriptive and thus does not 
impose on programs any particular criteria or approach. Rather, it is suggestive, pointing to a 
wide array of possibilities and outlining a model process that users of this guide should adapt to 
their own particular circumstances and priorities. 

A companion to this narrative guide is a set of survey instruments that are presently under 
development and attached as appendices. One is a suggested tool to allow programs to conduct 
qualitative assessments of their own programmatic and educational characteristics. Other surveys, 
also useful as institutional assessment tools, will have the added benefit of allowing individual 
programs to compare their own measures against other programs that have used the same or 
comparable instruments. 

It is important to note that this project is not a prelude to further accreditation of graduate 
programs, most of which are already accredited by state, regional, or local accrediting bodies. 
Nonetheless, the AAMC and the Liaison Committee on Medical Education (LCME) do 
encourage self-evaluation for the purposes of maintaining the quality of Ph.D. education, and thus 
this document was conceived of and prepared by the GREAT Group as a resource to its own 
membership. Input on this guide and other aspects of this project are welcomed and may be sent 
to the GREAT Group Executive Secretary by e-mail at <acshipp@aamc.org>. 
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II. 



The History and Mission of Graduate Education in Medical Schools 



For almost a century, medical schools have played an important role in graduate education, a role 
that was significantly expanded after World War II, when the federal government vested in 
academic institutions the predominant share of responsibility for conducting the national research 
effort and for training successive generations of researchers. This arrangement, largely the vision 
of Vannevar Bush, Director of the Office of Scientific Research and Development under the 
Roosevelt and Truman administrations, was successful because it integrated teaching and 
research. This allowed the educational process to benefit from the knowledge emerging from 
cutting edge scientific activity, while research benefitted from the exuberance, creativity, and 
effort of young scholars eager to learn and become skilled investigators. 

This paradigm has produced a highly successful system of training and has led to graduate 
programs as diverse as the institutions that house them. Many programs, including some in 
biomedical research, are based within schools of arts and sciences at colleges and universities, an 
arrangement that integrates graduate training within the broad tradition of scholarship at these 
institutions and allows undergraduates exposure to the research programs and graduate level 
activities of departments. While learning from their own mentors, graduate students in turn can 
develop their own didactic skills in serving as teaching assistants and instructors in baccalurate- 
level courses. 

Medical schools, on the other hand, are responsible for most of the graduate training that occurs 
in the biomedically-related sciences, a system that has grown out of the relationship between 
basic biomedical research and clinical activity. Research is an iterative process in which the 
findings of fundamental scientific investigations yield knowledge leading to new, more effective 
clinical practices, while clinical experience generates more questions that must be answered 
through basic research. Thus, the presence of a graduate program within the medical school 
allows doctorate students to appreciate the clinical relevancy of their basic science curriculum. 
Medical students, in turn, benefit from the presence of an active program of research since the 
latest research findings can be readily incorporated into their curricula, and they can be exposed 
to, and in some cases inspired to pursue, research careers. 

As a consequence of these benefits, medical school-based graduate programs have been highly 
successful. Ninety-eight percent of U.S. medical schools conduct Ph.D. training. In a 1997 
survey of these institutions, 8 1 schools responded and identified 629 individual departments and 
programs responsible for graduate education (by extrapolation, the total number may be as high as 
800-900). The same survey revealed that a majority — some 60 percent — of Ph.D. s in the 
biomedically-related sciences, in fact, emerge from medical school-based programs.' Graduates 
of these programs go on to become faculty, industrial researchers, and work in other capacities 
that serve the biomedical sciences. 

One way graduate programs ensure their success is to conduct self-assessments. This is a process 
by which programs regularly revisit, and if necessary redefine, their goals while looking toward 
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indicators that allow them to determine the extent to which those goals are being achieved. The 
concepts of programmatic and institutional diversity, as alluded to previously, are key. Each 
program fills a different niche, and programs should accordingly use varying criteria for self- 
assessment. Some programs may emphasize special features of the didactic experience, whereas 
others may focus on exposure to specialized research approaches. Some programs excel in 
producing clinical researchers, others basic researchers. Some are successful at training 
academics, others industry scientists. Thus, programs must discern their primary goals with 
regard to these and other objectives, and assess themselves accordingly. 

This guide is intended to aid programs in that process of self-assessment. The following section 
identifies goals and objectives commonly valued by graduate programs. Section IV outlines a 
process by which programs can achieve success that involves goal setting, self-evaluation, and 
efforts toward improvement. The process of “benchmarking” success is discussed in Section V, 
and a series of draft survey instruments and tables designed to facilitate the collection of 
benchmarking data are provided after the narrative section of this document. The final section 
points readers to other resources (e.g., articles, books, Web sites) that can shed further light on 
ways to conduct self-assessments. 

III. The Goals and Objectives of Successful Graduate Programs 

As a consequence of the diversity discussed above, graduate programs have different missions, 
goals, and objectives, and serve various student populations. This diversity lends in part to the 
success of these programs collectively. Nonetheless, there are some educational goals that 
virtually all programs aim to achieve. Most program directors agree that they intend to produce 
graduates who are: 

• Competent, skilled experimentalists, 

• Problem solvers, 

• Critical and independent thinkers, and 

• Expert in their field, with both depth and breadth of knowledge. 

In addition, programs aim to instill in individuals personal characteristics that are key to 
professional success in any career, leading to scientists who are also: 

• Leaders, 

• Excellent communicators, 

• Good mentors, nurturers, and teachers, 

• Organized administrators, 

• Exemplars of high ethical standards, and 

• Collaborators and team players. 

Finally, there are special characteristics that some programs may wish to see in their graduates, 
given the program’s special mission or setting. Specific programs may wish to produce graduates 
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who excel in techniques particularly relevant to a given discipline, seek to conduct patient- 
oriented research, or have other specific qualities, training, or goals. Moreover, various programs 
will seek to serve students who are primarily from underrepresented minorities, are state or 
community residents, are internationally competitive, or have other unique characteristics. 
Definitions of program success should encompass these objectives as well. 

Graduate deans and faculty judge their success not only by the quality and skills of the graduates 
they produce, but by the attributes of the program itself Indeed, some characteristics in particular 
seem to be common to programs that are successful. In a sense, these are the prerequisites of 
creating a faculty, a student body, and an environment consistent with excellence in education and 
the program’s overall objectives. Thus, a successful program is generally one that: 

• Has a faculty who are talented educators and capable researchers - Faculty 
must be skilled in pedagogic techniques, be excellent communicators, and through 
research, be current on the latest science in their discipline. 

• Provides adequate resources for teaching and research — Even the most 
talented and skilled faculty cannot educate effectively without the necessary 
instructional tools, library facilities, and research infrastructure, to which students 
also need access as part of the didactic experience. 

• Conducts quality research programs — Research and education go hand in hand, 
not only to assure currency and credibility on the part of faculty, but also to expose 
students to the research process, to conduct their own investigations, and to learn 
from their mentors. 

• Creates and fosters an intellectual environment and scientific probity — 

Students develop a sense of scholarship, integrity, and professionalism in large part 
through the examples set by those around them; thus fostering an environment that 
strives for the highest standards in all these areas is key. 

• Stimulates and promotes a quest for learning and a drive for creative inquiry 
- Good scientists are both creative thinkers and life-long learners who thirst for 
answers to unsolved problems and gaps in their knowledge base. 

• Retains good students and minimizes unnecessary attrition — Another 
indicator of a program’s success is its ability to retain the best students. 

Conversely, a large attrition of good students or a high occurrence of academic 
performance problems is undesirable. These characteristics may be signs that the 
program is not meeting in some way students’ expectations or not properly 
identifying applicants with adequate prerequisite training. 
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• Provides effective and efficient support of students’ needs -- Program 
administration has a responsibility to facilitate graduate education and career 
development. Services must be offered to students that permit identification of 
scholarships, access to constructive criticism and feedback, contacts with potential 
employers and fellowships, career counseling, and the provision of other services 
that benefit individuals while students or once launched into their careers. 

• Is valued by the faculty and central administration -- For a program to be 
successful, it must have a prominent place within the mission of the institution and 
enjoy political support from the faculty and administration. This characteristic is 
key to accomplishment and obtaining necessary resources. 

IV. How Programs Can Work Toward Achieving Success 

This document is predicated on the value of self-assessment, and thus review is a key activity in 
allowing programs to discern whether they are meeting their objectives and maintaining the 
standards that they have set for themselves or that they must meet to remain accredited. Review 
is, in effect, a circular process by which goals are defined, activities undertaken to achieve them, 
outcomes observed, and measures taken to assess progress. Informed by the knowledge that 
measurements allow, programs then revisit their goals and the process begins anew. 

Of course, outside entities will also be eyeing the extent to which programs achieve certain goals, 
including the parent institution, visiting consulting committees, NIH training grant reviewers, 
accrediting bodies, and the authors of independent reputational surveys published in the popular 
press to assist prospective students in the selection of programs. The presence of rigorous 
external review enhances, rather than diminishes, the value of self-assessment. The process of 
self-assessment will inevitably lead to improvements that will be benefit the program when these 
external evaluations are conducted. 

What follows is a general outline of the process of self-assessment that individual programs may 
choose to undertake. 

Step 1: Defining program goals and objectives — Programs must first consider the goals and 
objectives that they are trying to meet. Some of these may be universal in nature - those 
common to the educational and research missions that all programs share - and some may be 
specific to the particular niche that the program fills, as described earlier. The goals and 
objectives should relate to the type of student that the program hopes to attract, the training 
environment the program wishes to create, and the product the program intends to generate in 
terms of researcher competencies and career paths. The desired characteristics of faculty and the 
training environment should also be articulated. 
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Step 2: Pursuing activities that promote attainment of goals and objectives -- Successful 
programs - those that achieve their goals and objectives - generally undertake a common set of 
activities to enable their success. These include: 

Recruiting students who are a good match with its mission — This is achieved in part 
by communicating effectively to prospective students about the program’s mission and 
characteristics through literature and other recruitment tools. Providing data on career 
outcomes of graduates can also enable prospective students to assess if their own 
aspirations are in line with the achievements of the program’s graduates. 

Admitting applicants who will do well within the program — Programs use various 
criteria to identify students that seem to have the preparation and personal attributes that 
are required for success in graduate school. These may include GRE scores, GPAs, 
personal essays, undergraduate faculty recommendations, prior research experience and so 
forth, and the weight given to each criterion will vary depending on the program’s 
experience and goals. 

Providing the appropriate training — To ensure the success of enrolled students, a 
program must provide a comprehensive set of training experiences that not only includes 
course work and time at the laboratory bench, but that also permits opportunities to attend 
seminars, to develop communication skills, to observe the professional conduct of mentors 
and other professionals they may wish to emulate, and to receive appropriate levels of 
supervision and oversight. 

Establishing quality control of student progress and performance — Vigilant 
monitoring of student performance within the framework of programmatic expectations is 
necessary to ensure that standards are maintained and to identify when intervention may be 
necessary to correct deficiencies on the part of students. 

Providing guidance for careers — Once enrolled, students may change career directions 
as they are exposed to and engaged by new areas of science and professional activity. 
Mentors and advisors must be poised to work with students to understand the range of 
possibilities within the career choices they may make and to provide practical advice on 
achieving their goals. 

Recruiting effective faculty — The most effective faculty will naturally be those 
individuals whose talents, research interests, educational skills, and personal values best 
accommodate the program’s mission. Recruitment efforts should be structured to enhance 
the yield of applicants with these characteristics. Prior success as a faculty member is key, 
indicators of which may include history of research support, quality of publications, 
citation indices, communication and teaching experience, positions previously held, letters 
of reference, and other assessments. 
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Training to improve the pedagogical process - Once having recruited the right faculty, 
programs have a responsibility to assure their continued success as teachers. Assessment 
of faculty pedagogical skills and offering opportunities for faculty to improve their abilities 
as educators is a fundamental responsibility of any program. 

Step 3: Identifying indicators of success — Keeping in mind what the program is trying to 
achieve, program leaders should consider what factors may serve as reasonable indicators of 
program success. Many of them have been identified earlier in this guide and include quantitative 
measures that reflect both the backgroimd and preparation of individuals prior to becoming 
affiliated with the program (e.g., GRE scores, GPAs, etc.), as well as their achievements during 
and after. Other indicators may correspond to qualitative considerations, such as student 
satisfaction, producing scientists with a potential for leadership, creating a nurturing environment, 
and so forth. The process of identifying both types of indicators may involve faculty and student 
representatives, and as well as someone with the appropriate expertise to validate the correlation 
between the factors being measured or studied and the program characteristic that is being 
assessed. 

Step 4: Data gathering — Once a program has identified appropriate indicators of success, it will 
then have to obtain data on and assess the relevant characteristics of students, faculty, and other 
program elements as appropriate. Some indicators that programs may examine are identified in 
Tables 1 through 3 at the end of this document, grouped according to the aspect of the program 
being assessed. These tables focus specifically on assessment of faculty and program 
characteristics, graduate student populations, and recent graduates and their career outcomes. 
Shown in these tables are sources of these data within the institution as well as external sources 
that allow comparison to national statistics, where available. It should be noted that these tables 
focus largely on programs characteristics that are inherently quantifiable. 

Some characteristics of successful programs are highly qualitative in nature, aspects of which can 
nonetheless be assessed through quantitative means, such as the use of psychometric tools. Such 
qualitative characteristics include student satisfaction, environmental characteristics, integrity 
among student and faculty, and so forth. Some possible institutional mechanisms for assessing 
these characteristics include: 

• Surveys of students — with regard to the quality of faculty, the quality of their 
education, career goals, and the program environment. These results are 
potentially valuable on their own, or can be compared with similar surveys in the 
popular press or at other institutions. 

• Surveys of faculty — with regard to their teaching objectives, the quality of the 
student body, and the availability of needed resources for teaching and research. 
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• Self-evaluation questionnaires — these are targeted at the program directorship and 
serve to make leaders conduct critical evaluations of the state of their own 
programs. 

• Focus groups — conducted with either students or faculty to allow for 
identification and articulation of program strengths, weaknesses, and goals. 

As noted earlier, programs may wish to compare data on their own characteristics against those of 
other programs. In many cases, however, comparison data may not be current or readily 
available, may require significant manipulation to be useable, or may not provide the desired type 
of information about programs and their components. Therefore, it is hoped that the survey 
instruments found later in this document will help standardize approaches that programs use in 
collecting this data, facilitating the efforts of individual programs to compare their own measures 
against those of other programs that are similar in mission or scope. The survey instruments 
developed for this purpose are discussed in greater length under Section V of this document. 
Programs are also encouraged to establish their own benchmarks or thresholds based on their 
prior experience and current goals. 

Step 5: Implementing change — One important objective of assessing a program’s success is to 
identify those areas where success, however defined, is not being achieved and to correct the 
deficiencies or to overcome the obstacles contributing to this situation. Difficulty in achieving 
success may stem from any number of causes, including: inadequate financial resources, 
inadequate physical resources, failure to define realistic goals, poor evaluation of applicants, 
inability to recruit appropriate faculty, undesirable climate or ethics within the program, weak 
leadership, or many other factors that could be cited. Thus, for assessment to yield a productive 
outcome, programs must be poised to implement change to allow program success. 
Implementation of change is not a simple matter and there are a number of issues that programs 
will have to face in the process: 

• Administrative and financial resources - Implementation of curricular changes, 
recruitment of new faculty, development of new pedagogical tools, and other 
improvements will inevitably take resources which may not be readily at hand. 

Some resources will be financial and others will involve finding the information, 
skills, and tools necessary for the task. There may also be limitations to change 
based on what the administration of the parent institution is willing to accept, as 
well as restrictions placed on programs by state and regional accrediting boards. 

• Faculty involvement — It is also a human characteristic to resist change, and 
faculty may resist the implementation of new approaches unless there is a careful 
process of discourse, education, participation, and acceptance. The faculty must 
be in a position to recognize needed improvement and to accept the means for 
change, particularly since these measures may involve significant effort on their 
part. 
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• Cross-departmental communications — Many programs are interdisciplinary, 
involving several departments or even multiple institutions. Challenges related to 
resources, faculty involvement, and procedural matters may be complicated by 
these arrangements and by the additional administrative structures involved. 

V. Self-Assessment Surveys 

As previously noted, a companion to this narrative guide is a set of surveys to allow individual 
programs to conduct their own self-evaluations and to see where they stand relative to other 
programs that may have conducted the same data collection efforts. There are four elements to 
the survey project. The first two surveys, found in Appendix B, collect data at the institutional 
and program levels. The second set of surveys collect data on the qualities, objectives and 
satisfaction of individual students within the program. The results of these surveys can be 
compiled by individual programs and, once aggregated, may yield generalizable information on 
student populations. The specific survey topics are: 

• Faculty and Ph.D. program characteristics — The faculty are the core of any 
program and thus pivotal to its success. Consequently, it is worth looking at the 
numbers, demographics, credentials, and research experience of faculty, in addition 
to the resources and environment within which the program takes place. 

• Admissions and graduate student populations — Programs should look toward the 
qualifications, training, research experience, and demographics of incoming 
students, in addition to their progress as students, and their retention and attrition 
rates. 

• Student career goals and satisfaction — The student perspective is equally 
important, particularly as regards their view of faculty, coursework, research 
environment, and how the program enabled them to meet their career objectives. 
This document includes a survey for dissemination to individual students in specific 
graduate programs, as well as another instrument designed to allow reporting of 
aggregate data for comparison purposes. 

• Recent graduates — Faculty assessments of the intellect, skills, creativity, 
independence, and leadership qualities of the students they produce are important. 
These student characteristics are a measure of the faculty’s success in producing 
excellent researchers, and are addressed in the final survey instrument in this 
report. 

If programs using these surveys choose to compare data, the goal should not be to assess relative 
“quality” based on any single set of data. In fact, there is no reason to believe that, on any single 
criterion, one value is any better than another. Having a high faculty to student ratio, for 

9 




15 






example, is not inherently a “good” characteristic if faculty devote little time to students and 
teaching. So each variable will have to be assessed in the context of a particular program’s setting 
and goals. 

The surveys are found in the following section of this document. Individual programs should feel 
free to modify them in whatever way best meets their internal assessment needs. 



Endnotes 



' Ammons, S.W. and D.E. Kelly. “Profile of the Graduate Student Population in U.S. Medical 
Schools.” Academic Medicine, Vol 72, No. 9, September 1997. 
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Qualitative Questionnaire for Self-Evaluation 



of Graduate Programs 



External program review should generally be preceded by a process of self-evaluation in which 
the directorship examines the goals, strengths, weaknesses, and other features of the program. 

This kind of qualitative examination can be helpful in preparation for externally conducted 
evaluations as well as in providing points of focus for more detailed, quantitative self-assessments 
that might follow. Thus, this survey is a tool for program leaders to use for their own internal 
purposes in reflecting on the mission, organization, administration, and educational processes that 
are characteristic of the program. 

Mission and Organization 

1. What are your program’s goals and objectives and how were these determined? 

2. In terms of teaching and research, what are the program’s greatest strengths and greatest 
current needs? Are there important areas of the field not adequately represented in your 
department? 

3. How do the program’s strengths and weaknesses affect the graduate curriculum and 
laboratory training? 

4. What is the nature of research funding in your program (e.g., what share is hard money or 
soft?)? 

5. How does your graduate program substantially differ from other programs in major 
universities? 

Educational Process 

6. What is the process by which you revise your cumculum? By whom? Do students have 
input? 

7. When was the last time your curriculum offering and requirements at the graduate level 
were reviewed? How were curriculum changes implemented? 

8. Describe your admission process. How do you determine the target number of students to 
recruit? 

9. What faculty advising and mentoring is provided to graduate students in the first year and 
dissertation stages? 



10. Is there any mechanism for helping faculty become better mentors? 

1 1 . What are the teaching obligations of faculty? 

12. How do your students acquire skills for professionalism, ethical standards, and career 
development? Do you have a formal course on these subjects? 

13. If you could change one thing about your graduate program, what would it be if (a) no 
more resources were available, and (b) if you had new funds to achieve significant change? 

14. What is your student disciplinary mechanism? What are the mechanisms for dealing with 
violations of academic and research integrity? How is the student represented in the 
process? 

15. Do your students have formal channels of communication with the faculty? Is there a 
student organization? 

16. What opportunities do students have to teach or to conduct lectures? To what degree are 
students supported on teaching assistantships? 

17. Do you have a student exit interview? If so, how does it influence your program? 

1 8. Do you track your graduates in their careers? How do you track them? 



Program administration 

19. What kinds of resources and assistance does the program provide to faculty in carrying 
out their teaching and research responsibilities? 

20. How does the program, department, or institution reward faculty for effort applied to 
teaching? 

2 1 . How does the program assist faculty in obtaining external support for their research? 

22. In what other ways does the program administration contribute to the success of its 
faculty and students? 
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Assessment of Faculty and Ph.D. Program Characteristics 

Questionnaire 



Respondent Information 

Name of medical school: 

Name of person responsible for this 

survey on behalf of the medical school: 

Address: ^ 



Telephone number: t 'I - 

Fax number: t ^ - 

E-mail address: 

This survey instrument was developed by the AAMC Group on Graduate Research, Education, 
and Training to assist individual programs in the biomedical sciences in conducting self- 
assessments. Questions and comments on this instrument can be directed to the Group’s 
Executive Secretary at <acshipp@aamc.org>. 
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I. 



Characteristics of Ph.D.-awarding programs at your medical school 



A. How many programs do you have: 

B. How many programs are interdisciplinary: 

C. Do you have an M.D./Ph.D. program? 

Yes No 

II. Faculty characteristics 

A. Faculty to student ratios 

1 . a. How many graduate students were enrolled in your medical 

school’s Ph.D. programs as of October 1, 1998: 

b, How many first-year graduate students participated in your Ph.D. 
programs as of October 1, 1998: 

2. a. How many faculty participated in your graduate programs through 

teaching, advising, and committees as of October 1, 1998: 

b. How many basic science faculty participated in your graduate 

programs through teaching, advising, and committees as of October 
1, 1998: 

3. Estimate the number of the faculty cited in question II.A.2.a. that were 
eligible to be thesis advisors to students in your graduate programs: 



4. What percentage of the faculty in question II.A.3. have the following 
number of students in their labs: 

1 
2 

3 

4 

5 
>5 



100% 



.% 

.% 

% 

.%. 

.% 

% 
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Graduate program faculty characteristics 



1 . Percent of graduate program faculty who have a primary appointment in a 

Basic science department % 

Clinical science department % 

Other % 



100 % 



2. Number and percentage of imderrepresented minorities among all graduate 
program faculty: 

individuals % 



3. Number and percentage of women among all graduate program faculty: 
individuals % 



4. Indicators of national and international leadership in the scientific 
community 



Number of graduate program faculty in October 1, 1998 who were: 

Officers of scientific societies Member of honorary 

societies (e.g., NAS) 
and/or recipients of 
competitive fellowships 
(e.g., AAAS fellow) 

Scientific review body (e.g., Editorial boards 

NIH study sections) 

International honors 



5 . Over the last five years, has the number of your graduate faculty: 

risen 

declined 

stayed the same 



C. Faculty research 



Number of faculty involved in graduate education (from question II.A.2.a.) who 
had a grant in the previous 12 months from: 

1 . National Institutes of Health 

2. National Science Foundation 

3. Other federal sources 

4. Industrial sources (pharmaceutical, 

biotechnology, etc.) 

5. HHMI 

6. Other private foundations, voluntary health 

organizations, (e.g., American 

Heart Association) 

7. Other sources of external research support 

How many peer-reviewed articles were published in the previous 12 months that 
were authored by faculty affiliated with your graduate programs? 

Characteristics of your graduate programs 

A. Training support 

1 . National Research Service Award (NRSA) 

a. How many NRSA training grants do you have in all your graduate 
programs combined? 

grants 

b. How many NRSA-supported full time training positions (FTTPs) 
do you have in all graduate programs combined? 

FTTPs 

c. Do you have an MSTP grant? 

Yes No 

d. How many FTTPs do your graduate programs support with MSTP 
funds? 

FTTPs 

e. How many FTTPs are supported by NRSA fellowships? 

FTTPs 



2 . 



All sources of support 



a. Pooling all funds for stipend support at your entire medical school 
(including NRSA), what percent comes from; 

The NRSA program % 

Research grant assistantships % 

NSF fellowships % 

Other federal sources % 

Teaching assistantships % 

Institutional sources (non-TA) % 

Self (trainee) support % 

Industrial support % 

Foreign support % 

Other non-profit (i.e., foundations % 

and voluntary health organizations 
such as HHMI, AHA, etc.) 

B. Other characteristics 

f 

1 . a. Stipends: Do the graduate programs at your medical school offer a 

minimum stipend for all students? 

Yes No 

b. If yes, what is the amount? 

2. Do you offer tuition remission? 

Yes No 
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3. 



Benefits 





Health 


Maternity 


Dental 


Other 


a. Do graduate students at your 
medical school have access to 
the following benefits (check 
all that apply): 










b. For graduate students, does 
the institution contribute to 
premiums for the following 
benefits (check all that apply): 










c. Do dependents of graduate 
students at your medical school 
have access to the following 
benefits (check all that apply): 










d. For dependents, does the 
school contribute to premiums 
for the following benefits 
(check all that apply): 











4. Are graduate programs at your medical school accredited? 

Yes No 

If so, by what body (bodies) 
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Assessment of Admissions and Graduate Student Populations 

by Program 

Questionnaire 



Respondent Information 

Please provide the information requested on this survey individually for each of the graduate 
(Ph.D.) programs within your medical school. 

Name of medical school: 

Name of department or program to which this survey applies: 

Name of person responsible for this 

survey on behalf of the medical school: 

Address: 



Telephone number: t i - 

Fax number: ( ) - 

E-mail address: ■ 

This survey instrument was developed by the AAMC Group on Graduate Research, Education, 
and Training to assist individual programs in the biomedical sciences in conducting self- 
assessments. Questions and comments on this instrument can be directed to the Group’s 
Executive Secretary at <acshipp@aamc.org>. 
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I. Incoming Student Populations 

For students who matriculated into the program named above in the fall of 1998, please 
provide the following data: 

A. GRE scores (for the fall 1998) 

Are GRE scores required as a criterion for determining admissions into this 
program? Y es No 

Ph.D. 

If yes, please provide the following data: only M.D./Ph.D. 

1. Verbal: Median' score for matriculating class: 

2. Verbal: Minimum score for matriculating class: 

3. Quantitative: Median score for matriculating class: 

4. Quantitative: Minimum score for matriculating 

class: 

5. Analytical: Median score for matriculating class: 

6. Analytical: Minimum score for matriculating class: 

B. Undergraduate GPAs (for the fall 1998) 

1 . Mean GPA in science-related courses for 

matriculating class: . 

C. Diversity (as of the fall 1998) 



1. 


Total number of matriculating students (fall 98) 






2. 


Percentage of women in matriculating class: 


% 


% 


3. 


Percentage of underrepresented minorities: 


% 


% 


4. 


Percentage of students who are US citizens or 








permanent residents: 


% 


% 



D. Acceptance of offers 

1 . Number of completed applications to the 
program: 



' Median may best reflect the distribution of scores; other calculations may be used as well, such as mean or 
ranges. The intent is not to place excessive emphasis on standardized testing scores, which can have limited predictive 
value. 
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Ph.D. 

only 



M.D./Ph.D. 



2. Approximate percent of applicants who received 
offers of admission in the fall of 1998 (not 

including M.D./Ph.D. students): % % 

3. Approximate percent of those offered admission 
who actually matriculated in the fall of 1998 

(not including M.D./Ph.D. students): % % 



II. Development as a Scientist 

For students who graduated from your program in the spring of 1998, please provide the 
following data: 

A. Time to degree (defined as years [express with one decimal point] registered full- 
time in graduate school until completion of thesis, subtracting months on leave or 
in medical school): 

1 . Mean time to degree: . years 

B. Admission to candidacy^ 

1 . Do ail programs at your institution admit to candidacy in the same way? 

Yes No 

2. In your program, is admission to candidacy predicated on (check all that 
apply): 

satisfactory performance in required courses? 

demonstration of cumulative knowledge? 

preparation of an original research proposal ? 

3. a. Does your institution impose time limits on reaching candidacy 

status: 



Yes No 



^ Admission to candidacy means successful completion of all academic requirements preceding thesis and 

defense. 
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b. 



If yes, within what number of years must candidates reach this 
status? 



Years 



C. Degrees conferred 

1 . Number of Ph.D. degrees conferred in the previous academic year 

(summed over 12 months): 

2. Median age of these students (in question C.l) at receipt of Ph.D.: 

III. Attrition 

A. 1 . Percent of students who left the program in the past year without 

completing all degree requirements^: 

Ph.D. % M.D./Ph.D. % 

2. Percent of students who left in the past five (5) years without completing 
all degree requirements'*: 

Ph.D. % M.D./Ph.D. % 

B. Reasons given for leaving the program (per III.A.2.): 

Ph.D. only M.D./Ph.D. 





Male 


Female 


Male 


Female 


career changes^ 


% 


% 


% 


% 


relocation: 


% 


% 


% 


% 


poor performance: 


% 


% 


% 


% 


other: 


% 


% 


% 


% 


total 


100% 


100% 


100% 


100% 



^ Calculate as number of students who left in the last 12 months divided by all students matriculated in the 
program. Also include students on leave for greater than two years as of this date. 

“* Calculate as number of students who left in past five (5) years divided by the number matriculating in the 
same five-year period. 

^ Including decisions to switch to a different degree program or moving from a dual to a single degree program. 
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Other Aspects of the Graduate Student Experience 

A. Do students receive counseling from an advisor in first year? ^Yes ^No 

B. How often at a minimum must graduate students meet with the thesis committee? 
^times per year 

C. What percent of the students take an ethics curriculum: % 

D. Do you offer a course on career/survival skills? Yes ^No 



Survey of Student Career Goals and Satisfaction 



Respondent Information 

This survey is to be completed by individual students within each graduate program at the 
medical school. 

Name of student completing questionnaire (optional): 

Name of program in which you are/were enrolled: 

Name of medical school with which program is affiliated: 

Number of years you have been in the program: Years 

Have you graduated? Yes No 

This survey instrument was developed by the AAMC Group on Graduate Research, Education, 
and Training to assist individual programs in the biomedical sciences in conducting self- 
assessments. Questions and comments on this instrument can be directed to the Group’s 
Executive Secretary at <acshipp@aamc.org>. 



1. 



Do you plan on doing post-doctoral training: Yes No 

2. I ultimately plan on a career in (check all that apply): 



academia (teaching) academia (research) industry 

other (please indicate: ) don’t know 



3. A. In selecting my next professional position the most important considerations will 
be (rank order from 1 to 4); 

1 . professional development and satisfaction 

2. quality of life and family issues 

3. geographic location 

4. salary level 

B. With regard to my long-term career goals, the most important considerations are 
(rank order from 1 to 4): 



1 . professional development and satisfaction 

2. quality of life and family issues 

3. geographic location 

4. salary level 



4. 



How would you rate the following characteristics of your graduate experience 
(l=excellent, 5=very poor): 

Scale Icircle onel 

Excellent Very Poor 



a. Level, breadth, and content of courses 

b. Quality of instruction 

c. Mentoring/career guidance 

d. Research environment 

e. Program activities (e.g., seminars) 
f Interactions with fellow students 

g. Thesis advisory committee 

h. Program administration 

i. Mechanism for addressing grievances 

j. Overall level of satisfaction 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



5. Comments: 
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Aggregated Data on 
Student Career Goals and Satisfaction 



Respondent Information 

Name of person providing this aggregated, 
program-level data; ^ 

Title of the person named above: 

Name of program to which data pertains: 

Name of medical school with which program is affiliated; 

Address: 



Telephone number: ( i - 

Fax number: t i - 

E-mail address: 

Instructions 

This instrument can be used to aggregate the data from the Survey of Student Career Goals and 
Satisfaction, which will allow for analyses of program-wide data and comparisons with other 

programs that have used this instrument. 

This survey instrument was developed by the AAMC Group on Graduate Research, Education, 
and Training to assist individual programs in the biomedical sciences in conducting self- 
assessments. Questions and comments on this instrument can be directed to the Group’s 
Executive Secretary at <acshipp@aamc.org>. 
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Aggregated Responses to Survey on 
Student Career Goals and Satisfaction 









Students 




1st and 


3rd and 


in the 5th 




2nd Year 


4th Year 


Year and 




Students 


Students 


Beyond 


1. Percent of students planning on postdoctoral training 


% 


% 


% 


2. Percent of students planning a career in: 








A. academia (teaching) 


% 


% 


% 


B. academia (research) 


% 


% 


% 


C. industry 


% 


% 


% 


D. other 


% 


% 


% 


E. don’t know 


% 


% 


% 


3. A. Average ranking (to tenth decimal; e.g., 2.6) of 








considerations for respondent’s next professional 








position. 








1 . professional development and satisfaction 








2. quality of life and family issues 








3. geographic location 








O O 1^ 

4. salary level 








B. Average ranking (to tenth decimal; e.g., 2.6) of 








considerations for respondent’s long-term career 








goals 








1 . professional development and satisfaction 








1^ r 

2. quality of life and family issues 








3. geographic location 








o o 

4. salary level 
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Faculty Assessment of Recent Graduates 
Questionnaire 



Respondent Tnformation 

Name of faculty member completing questionnaire: 

Name of student being evaluated: 

Name of program with which student affiliated: 

Name of medical school: ^ ^ 

This survey instrument was developed by the AAMC Group on Graduate Research, Education, 
and Training to assist individual programs in the biomedical sciences in conducting self- 
assessments. Questions and comments on this instrument can be directed to the Group’s 
Executive Secretary at <acshipp@aamc.org>. 
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Please use a separate form for each student who graduated and rate him or her on a scale of 1 = 
excellent, 3 = satisfactory, to 5 = very poor. 

How would you rate this student with regard to the following characteristics: 

Scale (circle one') 



Excellent 



1 . Graduate coursework relative to other students? 1 

2. Commitment, drive, determination, and perseverance? 1 

3 . Creativity and imagination, in terms of experimental 1 

interpretation as well as design? 

4. Technical abilities? 1 



2 3 

2 3 

2 3 

2 3 



Very Poor 
4 5 

4 5 

4 5 

4 5 



5. Keeping up with and imderstanding the literature? 

6. Output and effectively completing tasks (e.g., 
translating observations into a presentable paper)? 

7. Ability to write clearly and persuasively? 

8. Ability to speak clearly and persuasively? 

9. Leadership qualities (in the lab and/or program)? 

10. Being well organized (good record keeping and 
well prepared lab notebooks) and effective time 
management? 

1 1 . Potential for independence (i.e., are they on a 
successful career trajectory)? 

12. Overall assessment as a productive scientist? 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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Existing Sources of Data for Graduate Program 
Self-Assessment 



Pending completion of the survey project, programs may wish to turn to existing sources of data, 
both internal and external, to benchmark their success in achieving educational and programmatic 
goals. The following tables present various characteristics that programs may wish to examine 
relative to their faculty, graduate student populations, and career outcomes of recent graduates. 



Table 1 

Assessment of Faculty and Program Characteristics 



Data 


Internal Sources of Data 


Sources of Comparison Data 


Publication rates of faculty 


Faculty evaluations 


PubMed/MEDLINE 


Citation statistics regarding 
faculty research 


Faculty evaluations 


ISI Citation Index, 

NRC Research Doctorate 
Programs in the United 
States: Continuity and 
Change 


Grant portfolio of faculty 

Cooperative research with 
industry 


Sponsored projects office 
Sponsored projects office 


NIH Web Site, WebCASPAR 


Awards and honors to 
program faculty 


Department office 


Honorific societies and 
academies 


Faculty/student ratios 

Demographic characteristics 
of faculty 


Program office 
Program office 


Peterson’s Graduate 
Programs in the 
Biological Sciences 
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Table 1 
(continued) 

Assessment of Faculty and Program Characteristics 



Data 


Internal Sources of Data 


Sources of Comparison Data 


Publication rates of faculty 


Faculty evaluations 


PubMed/MEDLINE 


Types and amount of training 
support (NRSA, TAs, 
RAs, etc.) 


Program office 


NIH, NSF 


Accreditation 


Program office 


Peterson’s Graduate 
Programs in the 
Biological Sciences 
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Table 2 



Assessment of Graduate Student Populations 



Data 


Internal Sources of Data 


Sources of Comparison Data 


GRE scores 


Dean of Graduate School 
Department Office 


Educational Testing Service 


Undergraduate GPAs 


Student applications 


Peterson’s Graduate 
Program in the 
Biological Sciences 


Honorary scholarships 


Fiscal officer, department 




Proportion of underrepresented 
minorities and women 


Student applications 


NRC Research-Doctorate 
Programs in the United 
States: Continuity and 
Change 


Length of prior research 
experience 


Student surveys 


Not available 


Time to degree 


Registrar’s office 


NRC Research-Doctorate 
Programs in the United 
States: Continuity and 
Change 


Retention/ Attrition 


Department 


Peterson’s Graduate 
Programs in the 
Biological Sciences 


First year GPAs 


Registrar’s office 




Graduate GPAs 


Registrar’s office 




"Admission to Candidacy" 
standing 


Program office 


Not available 
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Table 2 
(continued) 



Assessment of Graduate Student Populations 



Data 


Internal Sources Data 


Sources of Comparison Data 


Awards, fellowships, and 
honors received by 
students 


Program office 


NRC Research-Doctorate 
Programs in the United 
States: Continuity and 
Change 


Share of training grants from 
external scholarships 


Registrar’s office 


NRC Research-Doctorate 
Programs in the United 
States: Continuity and 
Change 


Student publication and 
citation rates 


Program office 


NRC Research-Doctorate 
Programs in the United 
States: Continuity and 
Change 


Number of degrees conferred 


Program officer 


NRC Research-Doctorate 
Programs in the United 
States: Continuity and 
Change 
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Table 3 



Assessment of Recent Graduates 



Data Internal Sources of Data 


Sources of Comparison Data 


Post-degree employment 
and placement statistics 


Development office 
Program office 


Commission on Professionals 
in Science and Engineering 
Survey of Doctorate 
Recipients 


Amount of external research 
support 




Federal agencies 


Involvement of graduates on 
national committees, panels, 
policy bodies 




Federal agencies, academies 


Receipt of special honors by 
graduates 




Not available 


Publication and citation 
statistics of graduates 




ISI Citation Index 


Career status well beyond 
graduation (e.g., 5 or 10 
years out) 


Development office, 
graduate survey 


Survey of Doctorate 
Recipients 
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Other Resources 



The literature and the Internet contain a number of resources that graduate programs may find 
useful as they embark on the task of assessing their own success. Some of the more notable 
works are cited below: 

Articles 



Holzemer, W.L. “Doctoral Education in Nursing: An Assessment of Quality, 1979-1984.” 
Nursing Research. Vol. 36, No. 2. March/April 1987. 

This article demonstrates how changes in the quality of graduate education in a particular 
discipline (in this case, nursing) can be assessed through questionnaires that capture data on 
quality-related program characteristics. Eighteen doctoral programs in nursing participated in 
1979 and again in 1984, providing data on faculty training and accomplishments, student ability 
and performance, resources, academic and social environments, program processes, and alumni 
achievements. Readers may cull fi’om this paper ideas about quantifiable characteristics of their 
own programs to assess. 



Kassebaum, D.G. “The Measurement of Outcomes in the Assessment of Educational 
Program Academic Medicine. Vol. 65, No. 5. May 1990. 

Although the context of this paper is medical education, rather than Ph.D. training, a number of 
parallels can be drawn with regard to assessment activities. The author explains why outcome 
measures have become an area of increasing emphasis in educational assessment, illustrates how 
they have been used, and provides a model linking various educational goals to quantifiable 
indicators. 

Books and Reports 

Association of American Universities Graduate Education Committee. Graduate Education 
Report Association of American Universities: Washington, D.C. 1998. 

This report, which describes the state of graduate education at AAU member institutions, includes 
a chapter on “best practices,” which is relevant to the task of self-assessment and program 
improvement. The chapter includes recommendations relevant to recruitment, admissions, 
graduate curriculum, mentoring, program evaluation, and other topics. 
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Commission on Professionals in Science and Technology. Road Map for Conducting 

Employment Surveys of Doctoral Graduates in S&E. CPST: Washington, D.C. 1998: 

This publication is intended to assist science and engineering professional societies and others in 
conducting surveys concerning the employment of recent doctoral graduates. It discusses survey 
design, data collection and management, and reporting. It may be of use to graduate programs 
that wish to conduct examinations of the career outcomes of their own graduates. 



Council of Graduate Schools Task Force on Academic Review of Graduate Programs, 

Academic Review of Graduate Programs. Council of Graduate Schools: Washington, 
D.C. 1990: 

This guide takes the reader through the process of program review. It is applicable to graduate 
programs in all disciplines and offers much useful and practical advice. 



National Research Council, Research Doctorate Programs in the United States: Continuity 
and Change. National Academy Press: Washington, D.C. 1995: 

This very thick volume provides a wealth of data on virtually all research doctorate programs in 
the country. Representative data elements include demographic characteristic of students, 
average time to degree, number of degrees conferred, citation statistics of program faculty, and so 
on. The 1995 report includes data collected in 1993. 



National Research Council. Summary Report 1996: Doctorate Recipients from United States 
Universities. National Academy Press: Washington, D.C. 1996: 

Data on recipients of research doctorates awarded by U.S. universities from July 1, 1995 through 
June 30, 1996 may be found in this report, which derives from the 1996 Survey of Earned 
Doctorates. It contains dozens of tables with statistics on the demographics, disciplines of study, 
sources of support and other characteristics of new doctorates. 
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Peterson’s Graduate Programs in the Biological Sciences. Peterson’s: Princeton, New 
Jersey. 1998: 

This publication, largely oriented to students exploring graduate school possibilities, provides a 
narrative description of each program that includes a few useful statistics as well, such as the 
numbers and demographic characteristics of faculty and students, average age of students, 
numbers of applicants and matriculants, minimum entrance exam requirements, and so forth. 



University of California Council of Graduate Deans. Excellence at Risk: The Future of 
Graduate Academic Education in the University of California. University of 
California: Oakland, California. 1997: 

The subject of this document is how the University of California system should build and sustain 
the highest quality graduate programs in the face of the many challenges confronting them. The 
recommendations may be applicable to other systems and include increasing academic 
specialization; rigorously measures of quality, productivity, and student outcomes; collecting and 
using data to inform decision-making; and attaining student body diversity. 

Internet Sites 

Association of Graduate Schools (AGS) 

The AGS is a body of the Association of American Universities (AAU) that brings together its 
graduate school deans. Its Web site includes data from a recent survey conducted to assess the 
steps that its membership has taken to modify graduate programs, including such issues as time- 
to-degree, program size, interdisciplinary programs, and career advising and placement. Access: 

http://www-ags.ucsd.edu/ags.html 



NIH Grants Page 

Data on individual grantees and principal investigators can be obtained at this site, and basic 
reports produced. Access: http://wvm.nih.gov/grants/award/award.htm 
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NSF Division of Science Resources 



This site allows access to tremendous amoimts of data, though prepared data briefs and Web 
CASPAR, from the Survey of Earned Doctorates, the Survey of Doctorate Recipients, and the 
Survey of Graduate Students and Postdoctorate Fellows in Science and Engineering. Also, 
federal grant data by institutions can be obtained and multiple cross-tabulations performed. 
Access : http ://www.nsf.gov/sbe/srs/ stats.htm 



Peterson’s Education Center, Graduate and Professional Study 

This page offers information on graduate programs across the U.S. as well as backgroimd 
information on graduate education issues. Access: http://www.petersons.com/graduate 



PhDs Org 

This site, which has a tremendous amoimt of information and links related to employment and 
careers for Ph.D.s, has a page specifically dedicated to ranking programs. The unique feature of 
this page is that it allows users to rank various characteristics of programs according to their 
importance, and then, drawing from data contained in the NRC publication Research-Doctorate 
Programs in the United States: Continuity and Change, the page orders programs according to 
the degree to which they fit the user’s stated priorities. Access: http://www.phds.org/ratings 
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